Development and evaluation of a prediction model for peripheral artery disease-related major adverse limb events using novel biomarker data.

OBJECTIVE: Prognostic tools for individuals with peripheral artery disease (PAD) remain limited. We developed prediction models for 3-year PAD-related major adverse limb events (MALE) using demographic, clinical, and biomarker data previously validated by our group. METHODS: We performed a prognostic study using a prospectively recruited cohort of patients with PAD (n = 569). Demographic/clinical data were recorded including sex, age, comorbidities, previous procedures, and medications. Plasma concentrations of three biomarkers (N-terminal pro-B-type natriuretic peptide [NT-proBNP], fatty acid binding protein 3 [FABP3], and FABP4) were measured at baseline. The cohort was followed for 3 years. MALE was the primary outcome (composite of open/endovascular vascular intervention or major amputation). We trained three machine learning models with 10-fold cross-validation using demographic, clinical, and biomarker data (random forest, decision trees, and Extreme Gradient Boosting [XGBoost]) to predict 3-year MALE in patients. Area under the receiver operating characteristic curve (AUROC) was the primary model evaluation metric. RESULTS: Three-year MALE was observed in 162 patients (29%). XGBoost was the top-performing predictive model for 3-year MALE, achieving the following performance metrics: AUROC = 0.88 (95% confidence interval [CI], 0.84-0.94); sensitivity, 88%; specificity, 84%; positive predictive value, 83%; and negative predictive value, 91% on test set data. On an independent validation cohort of patients with PAD, XGBoost attained an AUROC of 0.87 (95% CI, 0.82-0.90). The 10 most important predictors of 3-year MALE consisted of: (1) FABP3; (2) FABP4; (3) age; (4) NT-proBNP; (5) active smoking; (6) diabetes; (7) hypertension; (8) dyslipidemia; (9) coronary artery disease; and (10) sex. CONCLUSIONS: We built robust machine learning algorithms that accurately predict 3-year MALE in patients with PAD using demographic, clinical, and novel biomarker data. Our algorithms can support risk stratification of patients with PAD for additional vascular evaluation and early aggressive medical management, thereby improving outcomes. Further validation of our models for clinical implementation is warranted.

Predicting Outcomes Following Lower Extremity Endovascular Revascularization Using Machine Learning.

BACKGROUND: Lower extremity endovascular revascularization for peripheral artery disease carries nonnegligible perioperative risks; however, outcome prediction tools remain limited. Using machine learning, we developed automated algorithms that predict 30-day outcomes following lower extremity endovascular revascularization. METHODS AND RESULTS: The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity endovascular revascularization (angioplasty, stent, or atherectomy) for peripheral artery disease between 2011 and 2021. Input features included 38 preoperative demographic/clinical variables. The primary outcome was 30-day postprocedural major adverse limb event (composite of major reintervention, untreated loss of patency, or major amputation) or death. Data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 machine learning models were trained using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve. Overall, 21 886 patients were included, and 30-day major adverse limb event/death occurred in 1964 (9.0%) individuals. The best performing model for predicting 30-day major adverse limb event/death was extreme gradient boosting, achieving an area under the receiver operating characteristic curve of 0.93 (95% CI, 0.92-0.94). In comparison, logistic regression had an area under the receiver operating characteristic curve of 0.72 (95% CI, 0.70-0.74). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.09. The top 3 predictive features in our algorithm were (1) chronic limb-threatening ischemia, (2) tibial intervention, and (3) congestive heart failure. CONCLUSIONS: Our machine learning models accurately predict 30-day outcomes following lower extremity endovascular revascularization using preoperative data with good discrimination and calibration. Prospective validation is warranted to assess for generalizability and external validity.

Opioid-related deaths between 2019 and 2021 across 9 Canadian provinces and territories.

BACKGROUND: The drug toxicity crisis continues to accelerate across Canada, with rapid increases in opioid-related harms following the onset of the COVID-19 pandemic. We sought to describe trends in the burden of opioid-related deaths across Canada throughout the pandemic, comparing these trends by province or territory, age, and sex. METHODS: We conducted a repeated cross-sectional analysis of accidental opioid-related deaths between Jan. 1, 2019, and Dec. 31, 2021, across 9 Canadian provinces and territories using aggregated national data. Our primary measure was the burden of premature opioid-related death, measured by potential years of life lost. Our secondary measure was the proportion of all deaths attributable to opioids; we used the Cochrane-Armitage test for trend to compare proportions. RESULTS: Between 2019 and 2021, the annual number of opioid-related deaths increased from 3007 to 6222 and years of life lost increased from 126 115 to 256 336 (from 3.5 to 7.0 yr of life lost per 1000 population). In 2021, the highest number of years of life lost was among males (181 525 yr) and people aged 30-39 years (87 045 yr). In 2019, we found that 1.7% of all deaths among those younger than 85 years were related to opioids, rising to 3.2% in 2021. Significant increases in the proportion of deaths related to opioids were observed across all age groups (p < 0.001), representing 29.3% and 29.0% of deaths among people aged 20-29 and 30-39 years in 2021, respectively. INTERPRETATION: Across Canada, the burden of premature opioid-related deaths doubled between 2019 and 2021, representing more than one-quarter of deaths among younger adults. The disproportionate loss of life in this demographic group highlights the critical need for targeted prevention efforts.

Machine Learning to Predict Outcomes of Endovascular Intervention for Patients With PAD.

Importance: Endovascular intervention for peripheral artery disease (PAD) carries nonnegligible perioperative risks; however, outcome prediction tools are limited. Objective: To develop machine learning (ML) algorithms that can predict outcomes following endovascular intervention for PAD. Design, Setting, and Participants: This prognostic study included patients who underwent endovascular intervention for PAD between January 1, 2004, and July 5, 2023, with 1 year of follow-up. Data were obtained from the Vascular Quality Initiative (VQI), a multicenter registry containing data from vascular surgeons and interventionalists at more than 1000 academic and community hospitals. From an initial cohort of 262 242 patients, 26 565 were excluded due to treatment for acute limb ischemia (n = 14 642) or aneurysmal disease (n = 3456), unreported symptom status (n = 4401) or procedure type (n = 2319), or concurrent bypass (n = 1747). Data were split into training (70%) and test (30%) sets. Exposures: A total of 112 predictive features (75 preoperative [demographic and clinical], 24 intraoperative [procedural], and 13 postoperative [in-hospital course and complications]) from the index hospitalization were identified. Main Outcomes and Measures: Using 10-fold cross-validation, 6 ML models were trained using preoperative features to predict 1-year major adverse limb event (MALE; composite of thrombectomy or thrombolysis, surgical reintervention, or major amputation) or death. The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). After selecting the best performing algorithm, additional models were built using intraoperative and postoperative data. Results: Overall, 235 677 patients who underwent endovascular intervention for PAD were included (mean [SD] age, 68.4 [11.1] years; 94 979 [40.3%] female) and 71 683 (30.4%) developed 1-year MALE or death. The best preoperative prediction model was extreme gradient boosting (XGBoost), achieving the following performance metrics: AUROC, 0.94 (95% CI, 0.93-0.95); accuracy, 0.86 (95% CI, 0.85-0.87); sensitivity, 0.87; specificity, 0.85; positive predictive value, 0.85; and negative predictive value, 0.87. In comparison, logistic regression had an AUROC of 0.67 (95% CI, 0.65-0.69). The XGBoost model maintained excellent performance at the intraoperative and postoperative stages, with AUROCs of 0.94 (95% CI, 0.93-0.95) and 0.98 (95% CI, 0.97-0.99), respectively. Conclusions and Relevance: In this prognostic study, ML models were developed that accurately predicted outcomes following endovascular intervention for PAD, which performed better than logistic regression. These algorithms have potential for important utility in guiding perioperative risk-mitigation strategies to prevent adverse outcomes following endovascular intervention for PAD.

Temporal trends in hemodialysis access creation during the fistula first era.

OBJECTIVE: Although forearm arteriovenous fistulas (AVFs) are the preferred initial vascular access for hemodialysis based on national guidelines, there are no population-level studies evaluating trends in creation of forearm vs upper arm AVFs and arteriovenous grafts (AVGs). The purpose of this study was to report temporal trends in first-time permanent hemodialysis access type, and to assess the effect of national initiatives on rates of AVF placement. METHODS: Retrospective cross-sectional study (2012-2022) utilizing the Vascular Quality Initiative database. All patients older than 18 years with creation of first-time upper extremity surgical hemodialysis access were included. Anatomic location of the AVF or AVG (forearm vs upper arm) was defined based on inflow artery, outflow vein, and presumed cannulation zone. Primary analysis examined temporal trends in rates of forearm vs upper arm AVFs and AVGs using time series analyses (modified Mann-Kendall test). Subgroup analyses examined rates of access configuration stratified by age, sex, race, dialysis, and socioeconomic status. Interrupted time series analysis was performed to assess the effect of the 2015 Fistula First Catheter Last initiative on rates of AVFs. RESULTS: Of the 52,170 accesses, 57.9% were upper arm AVFs, 25.2% were forearm AVFs, 15.4% were upper arm AVGs, and 1.5% were forearm AVGs. From 2012 to 2022, there was no significant change in overall rates of forearm or upper arm AVFs. There was a numerical increase in upper arm AVGs (13.9 to 18.2 per 100; P = .09), whereas forearm AVGs significantly declined (1.8 to 0.7 per 100; P = .02). In subgroup analyses, we observed a decrease in forearm AVFs among men (33.1 to 28.7 per 100; P = .04) and disadvantaged (Area Deprivation Index percentile ≥50) patients (29.0 to 20.7 per 100; P = .04), whereas female (17.2 to 23.1 per 100; P = .03), Black (15.6 to 24.5 per 100; P < .01), elderly (age ≥80 years) (18.7 to 32.5 per 100; P < .01), and disadvantaged (13.6 to 20.5 per 100; P < .01) patients had a significant increase in upper arm AVGs. The Fistula First Catheter Last initiative had no effect on the rate of AVF placement (83.2 to 83.7 per 100; P=.37). CONCLUSIONS: Despite national initiatives to promote autogenous vascular access, the rates of first-time AVFs have remained relatively constant, with forearm AVFs only representing one-quarter of all permanent surgical accesses. Furthermore, elderly, Black, female, and disadvantaged patients saw an increase in upper arm AVGs. Further efforts to elucidate factors associated with forearm AVF placement, as well as potential physician, center, and regional variation is warranted.

Predicting outcomes following lower extremity open revascularization using machine learning.

Lower extremity open revascularization is a treatment option for peripheral artery disease that carries significant peri-operative risks; however, outcome prediction tools remain limited. Using machine learning (ML), we developed automated algorithms that predict 30-day outcomes following lower extremity open revascularization. The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity open revascularization for chronic atherosclerotic disease between 2011 and 2021. Input features included 37 pre-operative demographic/clinical variables. The primary outcome was 30-day major adverse limb event (MALE; composite of untreated loss of patency, major reintervention, or major amputation) or death. Our data were split into training (70%) and test (30%) sets. Using tenfold cross-validation, we trained 6 ML models. Overall, 24,309 patients were included. The primary outcome of 30-day MALE or death occurred in 2349 (9.3%) patients. Our best performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve (95% CI) of 0.93 (0.92-0.94). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.08. Our ML algorithm has potential for important utility in guiding risk mitigation strategies for patients being considered for lower extremity open revascularization to improve outcomes.

Vision Transformer-based Decision Support for Neurosurgical Intervention in Acute Traumatic Brain Injury: Automated Surgical Intervention Support Tool.

Purpose To develop an automated triage tool to predict neurosurgical intervention for patients with traumatic brain injury (TBI). Materials and Methods A provincial trauma registry was reviewed to retrospectively identify patients with TBI from 2005 to 2022 treated at a specialized Canadian trauma center. Model training, validation, and testing were performed using head CT scans with binary reference standard patient-level labels corresponding to whether the patient received neurosurgical intervention. Performance and accuracy of the model, the Automated Surgical Intervention Support Tool for TBI (ASIST-TBI), were also assessed using a held-out consecutive test set of all patients with TBI presenting to the center between March 2021 and September 2022. Results Head CT scans from 2806 patients with TBI (mean age, 57 years ± 22 [SD]; 1955 [70%] men) were acquired between 2005 and 2021 and used for training, validation, and testing. Consecutive scans from an additional 612 patients (mean age, 61 years ± 22; 443 [72%] men) were used to assess the performance of ASIST-TBI. There was accurate prediction of neurosurgical intervention with an area under the receiver operating characteristic curve (AUC) of 0.92 (95% CI: 0.88, 0.94), accuracy of 87% (491 of 562), sensitivity of 87% (196 of 225), and specificity of 88% (295 of 337) on the test dataset. Performance on the held-out test dataset remained robust with an AUC of 0.89 (95% CI: 0.85, 0.91), accuracy of 84% (517 of 612), sensitivity of 85% (199 of 235), and specificity of 84% (318 of 377). Conclusion A novel deep learning model was developed that could accurately predict the requirement for neurosurgical intervention using acute TBI CT scans. Keywords: CT, Brain/Brain Stem, Surgery, Trauma, Prognosis, Classification, Application Domain, Traumatic Brain Injury, Triage, Machine Learning, Decision Support Supplemental material is available for this article. © RSNA, 2024 See also commentary by Haller in this issue.

Impact of COVID-19 pandemic on prescription stimulant use among children and youth: a population-based study.

COVID-19 associated public health measures and school closures exacerbated symptoms in some children and youth with attention-deficit hyperactivity disorder (ADHD). Less well understood is how the pandemic influenced patterns of prescription stimulant use. We conducted a population-based study of stimulant dispensing to children and youth ≤ 24 years old between January 1, 2013, and June 30, 2022. We used structural break analyses to identify the pandemic month(s) when changes in the dispensing of stimulants occurred. We used interrupted time series models to quantify changes in dispensing following the structural break and compare observed and expected stimulant use. Our main outcome was the change in the monthly rate of stimulant use per 100,000 children and youth. Following an initial immediate decline of 60.1 individuals per 100,000 (95% confidence interval [CI] - 99.0 to - 21.2), the monthly rate of stimulant dispensing increased by 11.8 individuals per 100,000 (95% CI 10.0-13.6), with the greatest increases in trend observed among females, individuals in the highest income neighbourhoods, and those aged 20 to 24. Observed rates were between 3.9% (95% CI 1.7-6.2%) and 36.9% (95% CI 34.3-39.5%) higher than predicted among females from June 2020 onward and between 7.1% (95% CI 4.2-10.0%) and 50.7% (95% CI 47.0-54.4%) higher than expected among individuals aged 20-24 from May 2020 onward. Additional research is needed to ascertain the appropriateness of stimulant use and to develop strategies supporting children and youth with ADHD during future periods of long-term stressors.

Using machine learning to predict outcomes following suprainguinal bypass.

OBJECTIVE: Suprainguinal bypass for peripheral artery disease (PAD) carries important surgical risks; however, outcome prediction tools remain limited. We developed machine learning (ML) algorithms that predict outcomes following suprainguinal bypass. METHODS: The Vascular Quality Initiative database was used to identify patients who underwent suprainguinal bypass for PAD between 2003 and 2023. We identified 100 potential predictor variables from the index hospitalization (68 preoperative [demographic/clinical], 13 intraoperative [procedural], and 19 postoperative [in-hospital course/complications]). The primary outcomes were major adverse limb events (MALE; composite of untreated loss of patency, thrombectomy/thrombolysis, surgical revision, or major amputation) or death at 1 year following suprainguinal bypass. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). The best performing algorithm was further trained using intra- and postoperative data. Model robustness was evaluated using calibration plots and Brier scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, rurality, median Area Deprivation Index, symptom status, procedure type, prior intervention for PAD, concurrent interventions, and urgency. RESULTS: Overall, 16,832 patients underwent suprainguinal bypass, and 3136 (18.6%) developed 1-year MALE or death. Patients with 1-year MALE or death were older (mean age, 64.9 vs 63.5 years; P < .001) with more comorbidities, had poorer functional status (65.7% vs 80.9% independent at baseline; P < .001), and were more likely to have chronic limb-threatening ischemia (67.4% vs 47.6%; P < .001) than those without an outcome. Despite being at higher cardiovascular risk, they were less likely to receive acetylsalicylic acid or statins preoperatively and at discharge. Our best performing prediction model at the preoperative stage was XGBoost, achieving an AUROC of 0.92 (95% confidence interval [CI], 0.91-0.93). In comparison, logistic regression had an AUROC of 0.67 (95% CI, 0.65-0.69). Our XGBoost model maintained excellent performance at the intra- and postoperative stages, with AUROCs of 0.93 (95% CI, 0.92-0.94) and 0.98 (95% CI, 0.97-0.99), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.12 (preoperative), 0.11 (intraoperative), and 0.10 (postoperative). Of the top 10 predictors, nine were preoperative features including chronic limb-threatening ischemia, previous procedures, comorbidities, and functional status. Model performance remained robust on all subgroup analyses. CONCLUSIONS: We developed ML models that accurately predict outcomes following suprainguinal bypass, performing better than logistic regression. Our algorithms have potential for important utility in guiding perioperative risk mitigation strategies to prevent adverse outcomes following suprainguinal bypass.

Predicting Outcomes Following Endovascular Abdominal Aortic Aneurysm Repair Using Machine Learning.

OBJECTIVE: To develop machine learning (ML) models that predict outcomes following endovascular aneurysm repair (EVAR) for abdominal aortic aneurysm (AAA). BACKGROUND: EVAR carries non-negligible perioperative risks; however, there are no widely used outcome prediction tools. METHODS: The National Surgical Quality Improvement Program targeted database was used to identify patients who underwent EVAR for infrarenal AAA between 2011 and 2021. Input features included 36 preoperative variables. The primary outcome was 30-day major adverse cardiovascular event (composite of myocardial infarction, stroke, or death). Data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 ML models were trained using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve. Model robustness was evaluated with calibration plot and Brier score. Subgroup analysis was performed to assess model performance based on age, sex, race, ethnicity, and prior AAA repair. RESULTS: Overall, 16,282 patients were included. The primary outcome of 30-day major adverse cardiovascular event occurred in 390 (2.4%) patients. Our best-performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve (95% CI) of 0.95 (0.94-0.96) compared with logistic regression [0.72 [0.70-0.74)]. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.06. Model performance remained robust on all subgroup analyses. CONCLUSIONS: Our newer ML models accurately predict 30-day outcomes following EVAR using preoperative data and perform better than logistic regression. Our automated algorithms can guide risk mitigation strategies for patients being considered for EVAR.

Xylitol for the prevention of acute otitis media episodes in children aged 1-5 years: a randomised controlled trial.

OBJECTIVE: To investigate the regular use of xylitol, compared with sorbitol, to prevent acute otitis media (AOM), upper respiratory tract infections (URTIs) and dental caries. DESIGN: Blinded randomised controlled trial with a 6-month study period. SETTING: Enrolment took place at 11 primary care practices in Ontario, Canada. PATIENTS: Children aged 1-5 years who did not use xylitol or sorbitol at enrolment. INTERVENTIONS: Children were randomly assigned to use a placebo syrup with sorbitol or xylitol syrup two times per day for 6 months. MAIN OUTCOME MEASURES: Primary outcome was the number of clinician-diagnosed AOM episodes over 6 months. Secondary outcomes were caregiver-reported URTIs and dental caries. RESULTS: Among the 250 randomised children, the mean (SD) age was 38±14 months and there were 124 girls (50%). There were three clinician-diagnosed AOM episodes in the 125 placebo group participants and six in the 125 xylitol group participants (OR 2.04; 95% CI 0.43, 12.92; p=0.50). There was no difference in number of caregiver-reported URTI episodes (rate ratio (RR) 0.88; 95% CI 0.70, 1.11) between the placebo (4.2 per participant over 6 months; 95% CI 3.6, 5.0) and xylitol (3.7; 95% CI 3.2, 4.4) groups. Dental caries were reported for four participants in the placebo group and two in the xylitol group (OR 0.42; 95% CI 0.04, 3.05; p=0.42). In a post-hoc analysis of URTIs during the COVID-19 pandemic, the rate among the 59 participants receiving placebo was 2.3 per participant over 6 months (95% CI 1.8, 3.0) and for the 55 receiving xylitol, 1.3 over 6 months (95% CI 0.92, 1.82; RR 0.56; 95% CI 0.36, 0.87). The most common adverse event was diarrhoea (28% with placebo; 34% with xylitol). CONCLUSIONS: Regular use of xylitol did not prevent AOM, URTIs or dental caries in a trial with limited statistical power. A post-hoc analysis indicated that URTIs were less common with xylitol exposure during the COVID-19 pandemic, but this finding could be spurious. TRIAL REGISTRATION NUMBER: NCT03055091.

Grand rounds in methodology: key considerations for implementing machine learning solutions in quality improvement initiatives.

Machine learning (ML) solutions are increasingly entering healthcare. They are complex, sociotechnical systems that include data inputs, ML models, technical infrastructure and human interactions. They have promise for improving care across a wide range of clinical applications but if poorly implemented, they may disrupt clinical workflows, exacerbate inequities in care and harm patients. Many aspects of ML solutions are similar to other digital technologies, which have well-established approaches to implementation. However, ML applications present distinct implementation challenges, given that their predictions are often complex and difficult to understand, they can be influenced by biases in the data sets used to develop them, and their impacts on human behaviour are poorly understood. This manuscript summarises the current state of knowledge about implementing ML solutions in clinical care and offers practical guidance for implementation. We propose three overarching questions for potential users to consider when deploying ML solutions in clinical care: (1) Is a clinical or operational problem likely to be addressed by an ML solution? (2) How can an ML solution be evaluated to determine its readiness for deployment? (3) How can an ML solution be deployed and maintained optimally? The Quality Improvement community has an essential role to play in ensuring that ML solutions are translated into clinical practice safely, effectively, and ethically.

Using Machine Learning (XGBoost) to Predict Outcomes After Infrainguinal Bypass for Peripheral Artery Disease.

OBJECTIVE: To develop machine learning (ML) algorithms that predict outcomes after infrainguinal bypass. BACKGROUND: Infrainguinal bypass for peripheral artery disease carries significant surgical risks; however, outcome prediction tools remain limited. METHODS: The Vascular Quality Initiative database was used to identify patients who underwent infrainguinal bypass for peripheral artery disease between 2003 and 2023. We identified 97 potential predictor variables from the index hospitalization [68 preoperative (demographic/clinical), 13 intraoperative (procedural), and 16 postoperative (in-hospital course/complications)]. The primary outcome was 1-year major adverse limb event (composite of surgical revision, thrombectomy/thrombolysis, or major amputation) or death. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained 6 ML models using preoperative features. The primary model evaluation metric was the area under the receiver operating characteristic curve (AUROC). The top-performing algorithm was further trained using intraoperative and postoperative features. Model robustness was evaluated using calibration plots and Brier scores. RESULTS: Overall, 59,784 patients underwent infrainguinal bypass, and 15,942 (26.7%) developed 1-year major adverse limb event/death. The best preoperative prediction model was XGBoost, achieving an AUROC (95% CI) of 0.94 (0.93-0.95). In comparison, logistic regression had an AUROC (95% CI) of 0.61 (0.59-0.63). Our XGBoost model maintained excellent performance at the intraoperative and postoperative stages, with AUROCs (95% CI's) of 0.94 (0.93-0.95) and 0.96 (0.95-0.97), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.08 (preoperative), 0.07 (intraoperative), and 0.05 (postoperative). CONCLUSIONS: ML models can accurately predict outcomes after infrainguinal bypass, outperforming logistic regression.

Evaluation of Machine Learning Approaches for Predicting Warfarin Discharge Dose in Cardiac Surgery Patients: Retrospective Algorithm Development and Validation Study.

BACKGROUND: Warfarin dosing in cardiac surgery patients is complicated by a heightened sensitivity to the drug, predisposing patients to adverse events. Predictive algorithms are therefore needed to guide warfarin dosing in cardiac surgery patients. OBJECTIVE: This study aimed to develop and validate an algorithm for predicting the warfarin dose needed to attain a therapeutic international normalized ratio (INR) at the time of discharge in cardiac surgery patients. METHODS: We abstracted variables influencing warfarin dosage from the records of 1031 encounters initiating warfarin between April 1, 2011, and November 29, 2019, at St Michael's Hospital in Toronto, Ontario, Canada. We compared the performance of penalized linear regression, k-nearest neighbors, random forest regression, gradient boosting, multivariate adaptive regression splines, and an ensemble model combining the predictions of the 5 regression models. We developed and validated separate models for predicting the warfarin dose required for achieving a discharge INR of 2.0-3.0 in patients undergoing all forms of cardiac surgery except mechanical mitral valve replacement and a discharge INR of 2.5-3.5 in patients receiving a mechanical mitral valve replacement. For the former, we selected 80% of encounters (n=780) who had initiated warfarin during their hospital admission and had achieved a target INR of 2.0-3.0 at the time of discharge as the training cohort. Following 10-fold cross-validation, model accuracy was evaluated in a test cohort comprised solely of cardiac surgery patients. For patients requiring a target INR of 2.5-3.5 (n=165), we used leave-p-out cross-validation (p=3 observations) to estimate model performance. For each approach, we determined the mean absolute error (MAE) and the proportion of predictions within 20% of the true warfarin dose. We retrospectively evaluated the best-performing algorithm in clinical practice by comparing the proportion of cardiovascular surgery patients discharged with a therapeutic INR before (April 2011 and July 2019) and following (September 2021 and May 2, 2022) its implementation in routine care. RESULTS: Random forest regression was the best-performing model for patients with a target INR of 2.0-3.0, an MAE of 1.13 mg, and 39.5% of predictions of falling within 20% of the actual therapeutic discharge dose. For patients with a target INR of 2.5-3.5, the ensemble model performed best, with an MAE of 1.11 mg and 43.6% of predictions being within 20% of the actual therapeutic discharge dose. The proportion of cardiovascular surgery patients discharged with a therapeutic INR before and following implementation of these algorithms in clinical practice was 47.5% (305/641) and 61.1% (11/18), respectively. CONCLUSIONS: Machine learning algorithms based on routinely available clinical data can help guide initial warfarin dosing in cardiac surgery patients and optimize the postsurgical anticoagulation of these patients.

Predicting recurrent interventions after radiocephalic arteriovenous fistula creation with machine learning and the PREDICT-AVF web app.

OBJECTIVE: Kidney Disease Outcomes Quality Initiative (KDOQI) guidelines discourage ongoing access salvage attempts after two interventions prior to successful use or more than three interventions per year overall. The goal was to develop a tool for prediction of radiocephalic arteriovenous fistula (AVF) intervention requirements to help guide shared decision-making about access appropriateness. METHODS: Prospective cohort study of 914 adult patients in the United States and Canada undergoing radiocephalic AVF creation at one of the 39 centers participating in the PATENCY-1 or -2 trials. Clinical data, including demographics, comorbidities, access history, anatomic features, and post-operative ultrasound measurements at 4-6 and 12 weeks were used to predict recurrent interventions required at 1 year postoperatively. Cox proportional hazards, random survival forest, pooled logistic, and elastic net recurrent event survival prediction models were built using a combination of baseline characteristics and post-operative ultrasound measurements. A web application was created, which generates patient-specific predictions contextualized with the KDOQI guidelines. RESULTS: Patients underwent an estimated 1.04 (95% CI 0.94-1.13) interventions in the first year. Mean (SD) age was 57 (13) years; 22% were female. Radiocephalic AVFs were created at the snuffbox (2%), wrist (74%), or proximal forearm (24%). Using baseline characteristics, the random survival forest model performed best, with an area under the receiver operating characteristic curve (AUROC) of 0.75 (95% CI 0.67-0.82) at 1 year. The addition of ultrasound information to baseline characteristics did not substantially improve performance; however, Cox models using either 4-6- or 12-week post-operative ultrasound information alone had the best discrimination performance, with AUROCs of 0.77 (0.70-0.85) and 0.76 (0.70-0.83) at 1 year. The interactive web application is deployed at https://predict-avf.com. CONCLUSIONS: The PREDICT-AVF web application can guide patient counseling and guideline-concordant shared decision-making as part of a patient-centered end-stage kidney disease life plan.

Impact of the COVID-19 pandemic on antidepressant and antipsychotic use among children and adolescents: a population-based study.

Background: The COVID-19 pandemic was associated with increases in the prevalence of depression, anxiety and behavioural problems among children and youth. Less well understood is the influence of the pandemic on antidepressant and antipsychotic use among children. This is important, as it is possible that antidepressants and antipsychotics were used as a "stop-gap" measure to treat mental health symptoms when in-person access to outpatient care and school-based supportive services was disrupted. Furthermore, antipsychotics and antidepressants have been associated with harm in children and youth. We examined trends in dispensing of these medications two years following the pandemic among children 18 years of age and under in Ontario, Canada. Methods: We conducted a population-based time-series study of antidepressant and antipsychotic medication dispensing to children and adolescents ≤18 years old between September 1, 2014, and March 31, 2022. We measured monthly population-adjusted rates of antidepressant and antipsychotics obtained from the IQVIA Geographic Prescription Monitor (GPM) database. We used structural break analyses to identify the pandemic month(s) when changes in the dispensing of antidepressants and antipsychotics occurred. We used interrupted time series models to quantify changes in dispensing following the structural break and compare observed and expected use of these drugs. Results: Overall, we found higher-than-expected dispensing of antidepressants and antipsychotics in children and youth. Specifically, we observed an immediate step decrease in antidepressant dispensing associated with a structural break in April 2020 (-55.8 units per 1,000 individuals; 95% confidence intervals [CI] CI: -117.4 to 5.8), followed by an increased monthly trend in the rate of antidepressant dispensing of 13.0 units per 1,000 individuals (95% CI: 10.2-15.9). Antidepressant dispensing was consistently greater than predicted from September 2020 onward. Antipsychotic dispensing increased immediately following a June 2020 structural break (26.4 units per 1,000 individuals; 95% CI: 15.8-36.9) and did not change appreciably thereafter. Antipsychotic dispensing was higher than predicted at all time points from June 2020 onward. Conclusion: We found higher-than-expected dispensing of antidepressants and antipsychotics in children and youth. These increases were sustained through nearly two years of observation and are especially concerning in light of the potential for harm with the long-term use of antipsychotics in children. Further research is required to understand the clinical implications of these findings.

Predicting Major Adverse Cardiovascular Events Following Carotid Endarterectomy Using Machine Learning.

Background Carotid endarterectomy (CEA) is a major vascular operation for stroke prevention that carries significant perioperative risks; however, outcome prediction tools remain limited. The authors developed machine learning algorithms to predict outcomes following CEA. Methods and Results The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent CEA between 2011 and 2021. Input features included 36 preoperative demographic/clinical variables. The primary outcome was 30-day major adverse cardiovascular events (composite of stroke, myocardial infarction, or death). The data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 machine learning models were trained using preoperative features. The primary metric for evaluating model performance was area under the receiver operating characteristic curve. Model robustness was evaluated with calibration plot and Brier score. Overall, 38 853 patients underwent CEA during the study period. Thirty-day major adverse cardiovascular events occurred in 1683 (4.3%) patients. The best performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve of 0.91 (95% CI, 0.90-0.92). In comparison, logistic regression had an area under the receiver operating characteristic curve of 0.62 (95% CI, 0.60-0.64), and existing tools in the literature demonstrate area under the receiver operating characteristic curve values ranging from 0.58 to 0.74. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.02. The strongest predictive feature in our algorithm was carotid symptom status. Conclusions The machine learning models accurately predicted 30-day outcomes following CEA using preoperative data and performed better than existing tools. They have potential for important utility in guiding risk-mitigation strategies to improve outcomes for patients being considered for CEA.

Impact of a publicly-funded pharmacare program policy on benzodiazepine dispensing among children and youth: a population-based natural experiment.

BACKGROUND: In January 2018, the Government of Ontario, Canada, initiated a universal pharmacare program (OHIP+) for all individuals aged 24 years and younger. In April 2019, the program was amended to cover only children and youth without private insurance. Because benzodiazepines are commonly prescribed to children and youth despite their potential hazards, we examined whether changes in publicly-funded drug coverage influenced benzodiazepine dispensing trends in this demographic. METHODS: We conducted a population-based natural experiment study of benzodiazepine dispensing to children and youth in Ontario between January 2013 and March 2020. We used interventional autoregressive integrated moving average models to estimate the impact of OHIP + and its subsequent modification on these trends. RESULTS: The implementation of OHIP + was associated with an immediate increase in the monthly rate of benzodiazepine dispensing of 12.9 individuals per 100,000 population (95% confidence interval [CI]; 7.5 to 18.3 per 100,000). Benzodiazepine dispensing rates rose from 214.2 to 241.5 per 100,000 from December 2017 to March 2019, a 12.8% (95% CI 9.6-16.0%) increase. In stratified analyses, increases were most pronounced among females, children and youth living in the lowest income neighbourhoods and individuals aged 20 to 24. The April 2019 modification to OHIP + was not associated with changes in monthly benzodiazepine dispensing trends (0.39 individuals per 100,000; 95% CI -1.3 to 2.1 per 100,000). However, rates remained elevated relative to the period preceding OHIP + implementation. CONCLUSIONS: Implementation of a publicly-funded pharmacare program resulted in more children and youth being prescribed benzodiazepines.

Predicting survival post-cardiac arrest: An observational cohort study.

Introduction: Over 400,000 out-of-hospital cardiac arrest (OHCA) occur each year in Canada and the United States with less than 10% survival to hospital discharge. Cardiac arrest is a heterogenous condition and patient outcomes are impacted by a multitude of factors. Prognostication is recommended at 72 hours after return of spontaneous circulation (ROSC), however there may be other factors that could predict patient outcome earlier in the post-arrest period. The objective of our study was to develop and internally validate a novel clinical prediction rule to risk stratify patients early in the post-cardiac arrest period. Methods: We performed a retrospective cohort study of adult (≥18 years) post-cardiac arrest patients between 2010 and 2015 from the Epistry Cardiac Arrest database in Toronto. Our primary analysis used ordinal logistic regression to examine neurologic outcome at discharge using the modified Rankin Scale (mRS). Our secondary analysis used logistic regression for neurologic outcome and survival to hospital discharge. Models were internally validated using bootstrap validation. Results: A total of 3432 patients met our inclusion criteria. Our clinical prediction model was able to predict neurologic outcome on an ordinal scale using our predefined variables with an AUC of 0.89 after internal validation. The predictive performance was maintained when examining neurologic outcome as a binary variable and survival to hospital discharge. Conclusion: We were able to develop a model to accurately risk stratify adult cardiac arrest patients early in the post-cardiac arrest period. Future steps are needed to externally validate this model in other healthcare settings.

APPRAISE-AI Tool for Quantitative Evaluation of AI Studies for Clinical Decision Support.

Importance: Artificial intelligence (AI) has gained considerable attention in health care, yet concerns have been raised around appropriate methods and fairness. Current AI reporting guidelines do not provide a means of quantifying overall quality of AI research, limiting their ability to compare models addressing the same clinical question. Objective: To develop a tool (APPRAISE-AI) to evaluate the methodological and reporting quality of AI prediction models for clinical decision support. Design, Setting, and Participants: This quality improvement study evaluated AI studies in the model development, silent, and clinical trial phases using the APPRAISE-AI tool, a quantitative method for evaluating quality of AI studies across 6 domains: clinical relevance, data quality, methodological conduct, robustness of results, reporting quality, and reproducibility. These domains included 24 items with a maximum overall score of 100 points. Points were assigned to each item, with higher points indicating stronger methodological or reporting quality. The tool was applied to a systematic review on machine learning to estimate sepsis that included articles published until September 13, 2019. Data analysis was performed from September to December 2022. Main Outcomes and Measures: The primary outcomes were interrater and intrarater reliability and the correlation between APPRAISE-AI scores and expert scores, 3-year citation rate, number of Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) low risk-of-bias domains, and overall adherence to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement. Results: A total of 28 studies were included. Overall APPRAISE-AI scores ranged from 33 (low quality) to 67 (high quality). Most studies were moderate quality. The 5 lowest scoring items included source of data, sample size calculation, bias assessment, error analysis, and transparency. Overall APPRAISE-AI scores were associated with expert scores (Spearman ρ, 0.82; 95% CI, 0.64-0.91; P < .001), 3-year citation rate (Spearman ρ, 0.69; 95% CI, 0.43-0.85; P < .001), number of QUADAS-2 low risk-of-bias domains (Spearman ρ, 0.56; 95% CI, 0.24-0.77; P = .002), and adherence to the TRIPOD statement (Spearman ρ, 0.87; 95% CI, 0.73-0.94; P < .001). Intraclass correlation coefficient ranges for interrater and intrarater reliability were 0.74 to 1.00 for individual items, 0.81 to 0.99 for individual domains, and 0.91 to 0.98 for overall scores. Conclusions and Relevance: In this quality improvement study, APPRAISE-AI demonstrated strong interrater and intrarater reliability and correlated well with several study quality measures. This tool may provide a quantitative approach for investigators, reviewers, editors, and funding organizations to compare the research quality across AI studies for clinical decision support.